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PURPOSE AND DESIGN. OF THE EVALUATION ' - 



The Minnesota Evaluation, Project is a four-year project funded by 
the National Science Foundation (NSF) . Its purpose is* to evaluate in- 
service teacher training programs of five NSF Comprehensive Projects • 
During the second year (197,2-73) of the evaluation project, we started *^ 
to focus our evaluation on off-campus activities of the comprehensive ^\ 
pro ject^^ which are designed to facilitate the implementation or curric- 
ulum d^mnges in elementary ,^nd^ secondary, schools . One of these compo- 
nents is'^the portal school-. program at->the University of Wyoming. Other 
programs for the implementation of curricula have been developed b^ ,t^he 
other comprehensive projects and are being evaluated during the third 
year (1973-74) of the evaluation project. 

In this section of the repor^^we. describe the purpose and design of 
the evaluation. The following persons participated extensively in the 
evaluation study: 

• A ^ 

Glenn H. Bracht, Assistant Professor of Ed-ucational Pgy<!ihology 

at the, University of Minnesota, was director of the "evaluation. 

M. Donald Campbell, Research Assistant for the Minnesota Evalua- ' 
tion Project, was act^ely involved 'in all aspects* of the - 
study. ' ^ ^ * 

Harry Fehrenbacher , Research Assistant for the Minnesota Evalua-^ 
tion Project, w^s actively iftyolved in alL aspects of the 
^ stu^y. s ^' ^ ' ^ . 

V/, Todd Rogers, Sampling Coordinator of Research and Analysis 
for the National Assessment of Educational Progress, 
assisted in planning the evaluation ailcl was the. leader of ^ 
the secondary visitation team. 

Richard C. • Clark, Science Consultant for the Minnesota^^State 
Department of Education, was a -member of \he secondary 
visitation team. 

Roger T. Johnson, ^Associate Professor of Elementary Science 
Educat*ion at the University of Minnesota, was a member, of 
the Elementary visitation team* 




Purpose of the Evaluation • - ^ ^ 

/ 

' By the fall of 1972^ the University of Wyoming's por^tal ^school program 
had been in operation for about two years. .Hetfce.we felt there had been 
adequate opportunity for the staff to resolve some of the initial problems 
whi«h occur ^with the implementation of a new program. The timing seemed ^ 
appropriate, therefore, -for an evaluation of the potential useftfln^ss of 
the portal school concept as a delivery system for the implepaentation of 
curricula In elementary and secondary schools. The purposes of the 
evaluation were: 

1. To describe the portal school concept pnd identify- the key 

factors in the operation of. an effective portal school pTO- - ' 

gram. s • ^ ' , . 

* 2. To examine the coricerng about the portal school concept and 
judge the merits and shortcomings of the portal school pro- 
gram. 

3. To recommend ways in which a portal school program could 
operate effectively, , ' ^ 

The evaluation did not focus specifically on the effects of the University 
of Wyoming's portal school program; in fact, only four instances of portal 
schools were selected for the evaluation design. 

One reason for focusing on the usefulness of the portal school concept 
rather than on the effects of the Wyoming* program ^ was to satisfy several 
different audience^. Although the st'aff of the University dT^K^wiming , ' 
Compf^nsive Project may use this evaluation to modify the" operation of 
their portal* school program, the evaluation intended primarily for other 
institutions which might design de^livery- systems for the implementation of^ 
curricula. The evaluation is also significant for the NSF Division bf 
Precollege Education in Science, especially the Implementation Section of ' 
the Division. ' . ^ , . 

Evaluation &6sign . 

The major activities in the evaluation of the portal school program are* 
summarized in Table 1.^ The evaluation design was developed during the fall^ 
of 1972. Preliminary to work on the" design, Bracht visited the Wyoming . 
region in October tp interview bofct^ university and school district personnel - 
who have been involved in the portal' sthool program. In November, Robert 
S^ake consulted widh the evaluation pfpjec't and provided important ideas 
which helped in formulating the desigal * | , > 

' ■ • . 10 



ERIC 



'^ABLE 1 • 'I 

SCHEDULE OF ACTIVITIES IN THE^ -EVALUATION OF THE PORTAL SCHOOL PRObRAM 



' Date 



Activity 



October 1972 



October and 
c November 

Decembet 

January 4-5, 1^73 

January and 
February 

February 20-28 

March ^ 

March 27-30 

April 10-13 



April to 
July 



June 18 



V September 1973 



Bracht interviewed the coordinator for the, portal 
school program, science and mathematics super- 
visors who have established portal schools, and » 
portal leaders, " . * * 

Brkcht, Camp^lX, and Fehrenbachef desigtied the 
evaluation study. 

Evaluation plan was presented to the advisbry 
coitaiittee of the Minnesota Evaluation Projict* 

Site visitation was ra;^de to the medium-sized 
elementary portdl school by Bracht, Campbell, 
Fehrenbacher , and. Rogers, 

Questionnaires were developed by Bracht, Campbell, 
and Fehrenbacher, , ^ ' - 

Questionnaire^ were sent to^four portal schools. 

Questionnaire data were summarized. Plans were 
made for site visitations,* Content and format ' 
of interviews were developed, 

Site visitation was made to two elementary portal 
scho^qls and the University of Wyoming by Bracht, 
'Fehrenbacher, and Johnson • ' 

Site visitation was made to the two secondary ^ 
pprtal sch^ls by^ampbell, Clark, and Rogers, 

Evaluation report was written by Bracht, Campbell, 
and Fehrenbacher after receiving descriptions and 
judgm^ts from the two visitation teams. 

Oral rep.oxfe^was presented to the staff of the NSF 
Division of Precollege Education in Science in 
Washington, D,C, by Bracht and Clark, 

Reporta are being distributed to ' evaluation 
audiences. 
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The evaluation design placed high priority on a comprehensive descrip- 
tion of the pdrtal school program. This description was obtained through ' 
relatively full portrayals of all aspects of -four portal schools. .The pbr- 
^trayals included both university and school district involvement in the 
portal schools. These portrayals were prepared by examining materials and 
facilities, observing portal school activities, administering questionnaires 
and interviewing personnel in the program. ^ ^ ^ 

As the purpose of the. evaluatioy evolved, it seemed that the portal 
schools selected for portrayal shoulcf\represent specific types of portal 
schools. Rkther. than making a random Selection from the population of > 
portal schools in the Wyoming region , (Wyoming plus parts of five other 
states), we felt the purpose of the evaluation could be sa^sf led better * 
by selecting portal schools with the following characteristics : ^ 

\ / 1. Both elementary and secondary portal schools. 

2, Portal schools from both^arge and small school districts., 

3. Portal schools from both school dirti^ts with science 
supervisors and school districtsy«fithout .science supe^r- 

* \ visors. \ " X 

^ portal schoofc was obtained from the program cootdinator. 

The itifoVmatioif included the school district in which the portal school is 
located, date pf first portal school workshop,, whether elfementary or second- 
ary, content- of workshops (exploration, implementation; or creative ^xpan- 
slon)^ and degree of activity (]/nw, moderate, or high). 

We decided first to select ooiy active portal sdhools for portrayal. 
Although useful information could be obtained from the evaluation of 
relative-py inactive portal schools^ we felt that our limited resources- 
could be used i^ore effectively by including* four active portal schools 
in the design. Fur the rmore , niany members of the evaluation audience, 
would be interested in how to' estabi^ish siicc^fesful .portal school programs 
in their areas. Hence; information >bout what is working would be more 
useful than information about inactive portal schools. * 

* Since the cost of these portrayals would be .IjigK, only four portal 
schools ,were selected for portrayal. These we^e": ^ 



A. Ah elementary portal school in an urban school district / 
with Q'^.nnn QM1r^on^c • • f 



with 93,000 students 

B. An elementary portal school in a medium-^ized school 
district with 14,000 students. 

12 ■ ■ 
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C. a' secondary portal school in an urban school district 
wikh 93,000 students. ; , . 

D. A secondary portal, school in a sparsely populated region 

of approximately 17,000- square miles. This region includes 
eight school districts with a total of 11,000 students.^, 

0 

The two portal schools in the urbai^ area are in the sAme sdhool district. 
This school district has two fiiH-tlme sciance^^sj^rvisors . The medium- • » 
sized school district has a half-time science supervisor, but the position 
was established after the portal schooL began. In tile fourth -portal school 
there is^ a science supervisor in the school district where the ♦workshops- 
are conducted, but the other school districts in the region do not have ' 
science consultants. A unique featinre^^ this portal school is that it 
serves multiple school districts. VT 

Data Collection . ' ^ ^ 

f ' , • 

The portrayals of the foifr portal schoois^re based on ^ite visita- 
tions and questionnaires. During the site vis^ltations we interviewed 
teachers^ p.rincipals, portal Teader?, and science 'supervisees , observed 
teacher workshops,' and examined' materials and documents which^ relate to 
the portal school. 

\ ^ 

The initial site visitation was made to the elementary portal .school 

in the medixim-sized school district in January 1973^ Its^^purpose was to ^ 

gather preliminary data which would help us in constructing guestionnaires 

and planning later $ite visitations. The' visitation! team consistecj of ^ - 

Bracht, Campbell, Fehrenbacher , and Rogers. Rogers ^as inq^uded because y 

he would be the J'eader of the secondary visitation team in April. We 

interviewed ^oth teachers who had participated in port^Sschool workshops 

and teachers who had noE participated. Principals, poxfal leaders, the 

science supervisor, central administration officials, and ^th^ University 

.of Wyoming's field coordinator for the portal school prpgram were also 

interviewed. ' . - 

*■ ^ » 

Following this site visitation, questionnaires were developed for 
administration to te^ichers, principals, portal leader^s, and science super- , 
visors in the four portal schools. The domain for these questionnaires 
included the following topics: . * - * * 

1. Communication processes in this program. ^ ^ ' 

2. School district motivation for participation^ in the program. 
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3. School district commitment to the operation of the portal 
school. 

4. Content, processes, and effects of th6 porjial school work- 
shops. 

5. Availability of science curriculum materials in the portal 
school. ♦ 

,6. Selection and training of portal leadprs^. 

Specific questions were then written .from this domain for the various 
questionnaires. Although separate questionnaires were developed for 
teachers, principals,, portal leaders, and Science supervisors, th^re 
was some overlap of questions in the four forms. The^e'were also slight 
differences in the forms developed for elementary and secondary portal 
schools. Time did not permit pilot testing the questionnaires, but they 
were recycled about four to six times for revisions by the evaluation 
'team. * y - <: 

The proc^ure used fot selecting schools to receiv-e these question- 
nair^a varied for each portal school. Since we were not" evaluating 
specifically the four portal, schools bl!ft rather the usefulness of the 
portal School concept, we did. not insisjt on repreapntativeness of the 
"teacher and pringipal samples for the elementary portal schools. In the 
large urban school district, ten elementary schools were selected at . 
random, but one ^chool was liater r'emoved from the list, because it, had \ 
been closed. ' The elementary science supervisor then st^ggested three 
additional schools from which he felt it \jpuld be useful to gather data. 
Ten elementary schools were also randomly selected in the medium-sized 
school district, and the ^science supervisor then added one school from 
which he felt it would be usef ulT^a^gather dat^. For the secondary 
portal school in the large oirb^ district, however, the questionnaires 
•were sent to all Junior high schdol science teachers and principals-. ^ 
Only Junior high schools were sfel^cted because very few senioi^igh 
school .teachers have participated in the portal school. In The sparsely 
populated region, vthe questionnaires were sent to all Junior and senior 
high school science teachers and principals. All portal leaders aW 
science supervisors i^ the fou^ portal schoolp were sent the question-^ 
naires developed for them.' ^ ^ ^ ' 

In February the portal school questionnaires were mailed to the 
science supervisors, in the school districts with the four portal schools. 
They, in turn, distributed^ the questionnaires to t;he pQrtal leaders, 
principals, and teachers. One exception to this procedure occurred in 
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the secondary regional portal school where teacher and principal question- 
naires w^re mailed direct^ly to the principal. A questionnaire was also 
sent to the Uhiversity's coordinator of the portal school program for • 
information oa eacrh of the four portal schools. The questionnaire response 
*rate is summarized in Table 2. With the exception of the principal and, 
.teacher questionnaires in the urban school district, the quest;ionnaire data 
Vere analyzed prior to the final site visitations. - ^ . 

A science educator" was included on each visitation team for the final 
site Visits, Roger Johnson, an elementary science educator at the 
University of Minnesota, accompanied BraCht and Fehrenbacher to the elemen- 
tary 'portal schools, in March. Richard Clark, the scienpe consultant with 
the Minnesota^ State Department of Education, joined Rogers and Campbell on^ 
the secondary ^visitations. As during the initial visitation, science^ 
supervisors, portal- leaders V^principals, and teachers were interviewed. 
Teacher workshops were observed in all four portal schools. Documents used 
in the program, such as announcements, workshop handouts, and syllabi, 
were dlso collected. In Table ^2 we .have reported the nurfiber of schools 
visite^d, worjcshops observed, ahd. persons interviewed. 

" The interviews were not structured. Interviewers^ were encouraged to^ 
use a style that they find comfortable and' to explore and pursu^ informa- 
tion that might be yseful for the evaluation. HoweVer, some direction was 
given to the interviewers by defining sl»^ general areas of ^nf ozonation: 

1. Communication processes .between the univer^sity and the portal 
« schools and within the portal schools. 

^ - 2. Motivation and commitment of the school district to the portal 
school. 

' 3. Content aod processes of the portal school workshops. 

■ • , ; . ■ ■ 

4. Application of the science curricula by t€?achers in their 

clas^e's. ^ 
t . • • 

5. - Performanbe of portal leaders. 

6. Training of portal leaders.^ , * ^ • 

About 15 to 35 questions were developed for ^ach of these topics. These 
questions were not intended to be asted in e^ch interview but formed a 
pool of questions which^the interviewer gouJ^ use in 'planning and cbn- 
ducting the interview. Altfiough the use of the questions varied, the 
'interviewers obviously used them less frequently dairing the latter days 
of the visitations. - ' " ^ ' 



TABLE 2 



DATA COLLECTION SUMMARY 



Data Collection 
* 'Methods 



'Portal ^ ^ Portal 



School A 



School B 



Portal 
School C 



Portal 
'•School D 



Questionnaires:^ 
schools in sample 
Date 'returned 
Personnel surveyed^' 
Teachers 
Prii)cipals 
Ppjrtal leaders 
. y S.cience supervisors 

Site Visitations: 
Date of visit 
Schools visited 
Workshops observed ' ^ 
Personnel interviewed 
^Teachers 
•Principals' • 
Portal leaders 
Science supervisors 



■ 12 '• 


<■ I'l 


18 


i2 


Ape 11 


, narcn 


April 


tgffch 


iiy / ID^ 


/1 


lUU/ iz** . 




11/12 


11/11 


17/18 


12/12 


14/21 


3/3 


2/2 


3/3 


1/1 


l/l' 


1/1 


1/1 


March 29-30 


March 28 


^ April 10-il 


April 12 


7 


5 


5 


5 


2 


2 


■ , 1 


• 1 


33 


20'^' ■ ■ 


23 •'' 


' 28 


• 6 ■ ' 


3 


5* 




6. 


2 


2 " 


3 




1 

« 


1 


1' 



The j&ollowing code can be used to identify the 4)ortal schools:^ 
. Portal school'^^ Elementary ^ -urbari^ district 
Portal school B - Elementary, medium-sized district 
Portal school C - Secondary, uirbanf district , - ^ 

Portal scViool 1& - Secondary, sparsely populated region. 

The number to th^ left of the slash indicates the number of ques- 
tionnaires returned. The number to the right of *the slash indicates the 
number of questionnaires sent." 

^A preliminary site visitation was also made *to tWs school district 
on January 4-5 during which two schools were visited., ISfe^rvlews ^were 
conducted with nine' teachers, two principals, two portal ^Leaders, the 
science supervisor, two 'Central administration officials, plus thfe field 



coordinator for 4:he portal school program. 
^Includes assistarit principals. 
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Teachers, principals, and portal leaders usually were interviewed in 
their schools. Principals and portal leaders were interviewed individually 
inmost cases, but teacher^' were often interviewed in groups of two or 
three. Sometimes each member of the visitation team was conducting a 
separate interview while at other .times two or three interviewers were 
together? Science supervisors were^ interviewed several times duririg the 
visitation. ^ ^ 

The elementary visitation team also spent orm day at the University of 
Wyoming. The team interviewed the following persons about the design and 
operation of the portal school progfam: ^ ' 

1. Sam Harding, Director of the University of Wyoming Compre- ^ • 
^ hensive Project. 

2. Vincent Sindt, 'Coordinator of the portal school program. 

3. RoijaW Beiswenger, Paul Geisert, Robert Katisky, and 
Margariete Montague of the Science and Mathematics 
Teaching Center. . 

4. Gerald Meyer, Dean of the College of Arts and Sciences in 
. which the Science and Mathematics Teaching Center is ^ 

located. 

5. Paul Kepper and John Christopher of the Extension Division. 

The reader is cautioned that our information about the portal school 
program was obtained primarily from four portal schools. While it is 
better to portray four portal schools thari otfe, we do not know whether 
similar judgments would, hafve been madp about the portal school program if 
a different sample of, four portal schools had been selected for the study. 
On the other hand, the coordinator of the port^fl sthool program and the 
other members of the Scl^ence and- Mathematics Teaching Center examined a 
preliminary draft of the, evaluation report and found only one major 
difference between the findings Vf our four portrayals and their experi- 
ences with the entire program. . 
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UNIVERSITY RESPONSIBILmiS^IN. THE PORTAL SCHOOL PROGRAM 



We have identified three major categories of university responsi- 
bility which are in^ortant* factors in the portal school program: 

1. Organizing university resources, 

. 2. Establishing portal schools^ 

3. Training, certifying, and supporting portal leaders. 

While these responsibilities do not exist independently, of ^the schools 
districts, we think the major responsibility for their effectiveness 
lies at the university. 
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ORGmZING UNIVERSITY RESOURCES 



The Science and Mathematics Teaching ^Center (SMTC) of the University 
of Wyoming is the focal point of the portal school program. The SMTC was 
established to improve the .preservice and inservice training of science 
and mathematics teachers.' The inservice component is performed primarily 
through the portal school program. . The center contains curriculum mate- 
rials and supplies and has facilities for working wil:h materials and 
demonstrating their use to teachers. In addition, faculty members con- " 
duct research ajid development activities on various problems of science 
and mathematics/instruction and the leamii;ig process, >• 

The SMTC i^^eives its operating budget through the College of Arts 
and Sciences, b^t is is staffeM jointly by the College of Arts and Sciences 
and the Cplleg^ of Education, JUie courses of ffered by th6 SMTC, both pre- 
service and portal school courses, are listed in the Department of Natural 
Science. This department does not have a staff or budget and exists 'only 
as a mechanism for offering courses in science education. This arrange- 
ment provided the SMT&iwith the opportunity of defining courses for' the 
portal school program so they^uld meet the needs of elementary and sec- - 
ondary science teachers in tiielt specific situations, 

A major feature^ of the course descriptions is the flexibility of 
content which may be presented in the course. , Hence different elementary 
workshops with the s^me course number may cover, for example, (1) explora- 
tion of the ESS, SCIS, and SAPA curricula, (2^ exploration of the ESS and 
SCIS curricula, (3) implementation of the SAPA curriculum, (4) implementa- 
tion of the SCIS curriculum, etc. The courses offered through the 
Department of Natural Science are listed in Table 3. Another aspeqt of 
the flexibility is that courses are defined with variable" credit. 

A second major feature of the course <}esctiptions is the emphasis 
on application in the science class. The descriptions coavey the idea 
that an ele|entary or secondary teacher who has* had successful experi- 
ence in using the curriculum materials would be most qualified to teach 
the course. This idea coincides, of course, With the portal school 
design—using local teachers rather than, university professors as V 
portal leaders. \ .\ , 

. The SMTC carries out all of the financial transactions associated 
with the portal school program. It collects the tuition from workshop ; 
participants and makes ^disbursements to the extension division (for 
course registration and recording on transcript), portal leaders (for 
salaty), reserve fund (for subsequent pbrtal school activities)* and 
the revolving fund of the SMTC. The last section of this report contains 
more information about the 'financing of the portal school program. 



19 





^ TABLE 3 


COURSES 


OFFERED IN DEPARTMENT OF NATURAL SCIENCE 


Course 
Number 


Course Title 


669M 


Science for .Elementary School Teachers 1 


670M 


Physical Science for Secondary School feachers 


671M 


Biological. Science for Secondary School 
Teachers < 


869M 


Seminar in Science for Elementary School 
Teachers \ 

I 


870M " 


9 

oemxnar in rnysicai oclences tor Secondary 
School Teachers 


876M 


Investigations in Natural Science for 
El'ementary Teachers ' ^ 


877M • 


Inv^tigations in Natural Science 'for 
Secondary Teachers ; , 




^ 7^^ — ^ ~" ^ — 

4 

* * • 
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Aiyiough the SMTC has designed and established the portal school 
program without some of the traditional constraints of university policy 
and procedures, it is important that the center coordinate its activit>ies 
with* other university units. Since the SMTC has benefitted from s^ome 
exceptions to university policy- and procedure in the ways it receives 
and spends tuition mpney, certifies instructors (portal leaders); and 
conducts course registration, conflicts have occurred wiih the extension 
division and the College of Education. Problems of this sort should be 
resolved in order to maintain effectively the portal school program. 

^ IXi many universities, the several faculty members in an area of 
specialization, e.g.^ science education, generally function indepen- 
dently of each other. Beyond teaching assignments, each faculty member- 
decides, for the most part in what ways to be involved in disciplined 
Inquiry, service, and governance. The SMTC has had the effect of bringing 
tagether the faculty in science and mathematics education* to work on a 
dommon mission. As a group they usually reach consen^^i?^ about high 
priority needs on. which they can focus their activiti'e^"" While individ- 
ual faculty members still have the opportunity to decide on ways of * ' 
personal"" involvement, they can now coordinate their disciplined inquiry 
and -Service activities with .other faculty members. We think strong 
faculty commitment to a coordinated and focused program is essential for 
a successful portal school program. 'While it may be possible -to have 
thTs coordination and focus without a formal center like the SMTC, there . 
seem to be some advantages in having the .formal organization: 

^ 1. The c^ter gives the, faculty member a stronger identity with 
a prograA. Departments are organizational units for admin- 
istrative purposes and hence provide the faculty member 
wltlv-^ft artificial sense of identity. ^The faculty member 
proi)ably obtains greater' satisfaction from his identity 
with a program, e.g., science education, » rather than with 
an administrative unit, e.g., elementary education. The 
advantage of a formal program organisation, e.g.. Science 
Teaching Center, oVer an informal program designation, e.g., 
science education, is. that the formal center is probably ' 
more explicit in defining its goals, and hence the faculty 
.member is more likely to make a strong commitment for 
^ il^olyement in the portal school program* 

2. The center 1^ mote organized than an informal program desig- 
nation and tends to focus in a coordinated way on activities 
which support the portal school program. The formal center' 
is generally more efficient arid productive. 

3* A Renter can achieve considerably more visibility. The - 
success of the portal school program has givei;i the SMTC 
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substantial visibility in the university's central adminis- 
tration. This has implications, of course, for budget and 
other decisions. We. were also told that the SMTC, through 
the portal school program, has impressed a number of state 
legislators. 

While some type of formal organization may .be desirable for univer- 
sities to establish successful portal school programs, that obviously is* 
not sufficient. The organization consists of people and they must be 
« strongly committed to its goals. Merely substituting a formal organiza- 
tion for an informal group may^^je counter-productive. ' 
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' ESTABLISHING PORTAL SCHOOLS 

The portal school program is designed to deliver the services of 
the' SMTC to the school districts by establishing portal schools. The 
first s^p in flelivering- the program to the school districts is public- 
ity. The SMTC relies heavily on -its field coordinator to publicize the 
program. He is well knovm in the Wyoming region, having served as 
science consultant, fior both , the Wyoming and Colorado State Departments 
of EducatiX)n. His previous relationships with many school administratofi 
.and science teachers has facilitated his efforts to present the program 
to school districts. '"f"'* - 

Another means of .publicity occurs when enthusiastic participants 
tell others about the ^i.rorg;-am. One of the school districts visited was 
introduced to the portal school program in this way. The field coordi- 
.nator reported that other districts have also contacted the SMTC after 
receiving information about the program from colleagues. Finally, the 
SMTC publishes a newsletter which is distributed widely tAscience 
teachers and schools in the Wyoming region. 

The university.' s second step in establishing a portal school is to 
facilitate school district planning for the program. Again the field 
coordinator assumes- primary responslbild-ty . He visits with the science ■ 
supetyisor or school district administrators to discuss the district's 
science instruction needs. If both he and the school district" adminis- 
trators agree that a portal school could be an effective- ihstrument for 
curriculum change, the school district must then- develop a 'sfement of 
its goals for thT portal scl^ool. In one of the portal schools we visited, 
this initial' contact with the central administration and the statement 
of goals was not made. Instead, the program was developed by the SMTC 
'and the teacjhers. >As a result, it did not seem to be directed strongly 
toward any stl^ool (^strict goal and did not receive strong support from 
the school district's central administration. i 

Assessing school district* needs in science instructio;i' is an essen- 
tial stage in estajblishing a portal school. We suggest that it be dotie- • 
in a systematic way with input from teachers, principals, and others ■ 
with a responsibility for curriculum and instruction. The lieeds assesae 
ment should be planned; it cannot be done adequately in a two-hour ' 
meeting. It is the university's responsibility to stimulate the schocQ. 
district in assessing its needs and to suggest strategies for conducting 
the assessment. In some cases the needs assessment ha? already been 
done before the field coordinator starts working wifh the district. ' 

It is useful to classify the school district needs into the cate- 
gories of exploration, implementation , and creative expansion as the 
field Coordinator has been doing. This leads then to formulating go.als 
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and a portal school plan fqr the district. The^ university should examine 
the plan and help the district to clarify, its intents. The university 
should emphasize that the portal school is the school district's program 
for^responding to its science instruction' needs . It is not a program 
wh^e the university provides instructional services directly for te^achers' 
needs and interests, but it is -a program where the school district can use 
the resources of the university to provide an inservice program which meets 
the district's needs. ' 




After assist^,tvg-^he school district in assessing its needs and formu-. 
lating a general portal school plan, the university should seek a commit- 
mei^st. from the district to support* the portal school. This Includes a 
commitment to »both leadership and financial support from 'the central 
administration. Someone in the district should be designated? with the 
responsibility of directing the program and funds should be committed' 
for purchasing instructional materials. The field coordinator must (hen 
decide whether the school district has made a commitment to an adequate 
level of support for a successful portal school. 

Then the fie!ld coordinator helps the school district to select 
portal leader candidates. He emphasizes that the portal leader candidate 
should be a' teacher who^as been successful in teaching science and is 
interested in helping other teachers to improve thAiri science instruction. 
The leadet must^e abl^ to develop rapport with felio* teachers and dis- 
play enthusiasm for science instruction. After the field coordinator and 
the school district reach, consensus on a teacher\s potejitial, that teacher 
is aslced to-becoiijg^a portal leader candidate. Larger school di^tiricts 
usually have a science supervisor who assumes primary responsibility for 
selecting portal leader candidates. The candidate, however, does no(^ 
become a portal leader until certified by'the.SMTC. The leadership 
training and certifying functJoFts are describe'd.ln the next section. 
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TRAINING, CERTIFYING, AMD SUPPORTJNG PORTAL LEADERS 

• • ' ^ •■ « 

The minltute is the term used to describe, the leadership training 
institute for portal leaders. The minitutes are conducted in J:he SMTC ^ 
and consist of one to six wellcs (usually three weeks) of interaction 
between portal leader candidates and the faculty and material^^f the 
center. The purpose of the .minitjjte is to prepare the portal leader 
to conduct workshops to meet the specific needs of hiS school aistrict. 
The por^l leader receives graduate credit through tlie Department of > 
Natural Science for his participatiori. " ' [ 

. The content studied at the^ minitute varies with^tl^ needs of the , > 
candidate and the needs of his school district, but ge^ieralAy include^s 
the following ^oplcs: (1) teaching strategies, with spec^iaF^ emphasis 
^iven to the philosophy and use of the inquiry approaclj^; [ (2) curriculum 
materials, with emphasis on the' materi-als produced by NSF curriculum 
development projects; (3) skills in inter-personal .relations; and 
(4) strategies for, planning and conducting workshops. 

The minitute. follows a "come, help yourself" theme and places the '* 
candidate in an unstructured, self-directed environment. The portal 
leader candidate comes to the minitute with an awareness of the' school 
district's needs and its general portal schopl plan, ^fter being 
informed about the faculty, materials, and' facilities tfiat' are available, 
the -candidate is encouraged to strike out on his own in^eparation for 
his portal leaddr responsibilities. Attendance *at seminars, films, 
demons trat ions ,y and group activities is optional. ' The pnly^ specific 
assignment is that the candidate must produce ""a^detalled wbrkshop pl^in 
'for the school district's portal school. The field coordinator and 
other faculty members evaluate the plan and help^ the candidate to locate 
and develop the resources which are needed tp conduct th6 v^rkshop. 

The faculty of the ^SMTC described two advantag'es 1^ the uns^ruc- 
turedness of the minUute. The first relates to opportunity for indi- 
vidualization. The candidate must develop a workSlip plan for a specific 
school and then prepare himself to conduct it. Since, there are unique^ 
features in the needs of ea^ school disti;ict ai\{i ^ach &aildidate, the 
minitute must be unstructured to some extent. ^ ^'^ 

The faculty also claims that the unstructured environment provides 
a good test of whether the candidate 'will be> an effective -portal l^ad-er. 
The candidate is required (1) to take the initiativ^ in developing a 
course, (p to seek imt and use appropriate resources in developing a 
course, (3) to produce a course plan with meai;iingful content and struc- 
ture, and (4) to wotk inde^)endently when adequate resources are aVailabfe. 
The few participants .in the minitute who have-niOt measured up to the (pv 
criteria have recognized their weaknesses and voluntarily withdrawn as 
candidates for portal leaders. % 



Many of the portal leaders exprejssed dissatisfaction with tl(e unstruc- 
tured environment in the minitute. \fhen asked in an 'open-ended question 
about modifications they would recomniend in the minitute, about one-third 
volunteered that more structure would have been helpful. They a^so suggested 
better organization of the eContent presented in the minitut;e ^^tld more guid- 
ance in planning their own workshops* Suggestions weje also made for m6re 
materials and more time at the minitute. On the other hand, many' portal 
leaders reported that the uristruoturedness was very useful to them. Perhaps 
the degree of structure should .be contingent .upon the individual need for 
it. . , • 

Concerns about the content and structure of the minitute and. the certi- 
»Mca^ion process were expressed by a number of people we interviewed. Since 
these concerns surfaced late in our data collection, we do not have adequate 
information for making, judgments about these issues. This area should 
receive additional study before other institutions plan to implement portal 
school programs J 

One of the portal schools is conducting a minitute within the school 
district during the summer of 1973. Instead of transporting ^its teachers 
to the university for the leadership training, this school district is 
"importing some^of the SMTC faculty and materials in addition to using some 
of its experienced portal leaders, as staff for the minitute* This minitute 
will be more structured than thosfe conducted at the university. While the 
SMTC feels that the move to a school district site sacrifices the richness 
of the resources available to the portal -IfeTader candidates, the school 
district feels that the convenience and economy of having the minitute 
"at home" will increase the number of participants. , ■ ' 

After the workshop plan has been approved and the portal leader has 
been certified, he is prepared to conduct portal school workshoj^s. One of - 
the^hool districts we visited has added the requirement that the portal 
ieMer must use the curriculum materials in his class before conducting 
a, workshop about the curriculum^ The other school districts have ac>tepted 
the university certification as sufficient evidence of competency, 

\ 

After certification^ there is little, observation of the pgttal 
leader's activities by the SMTC. The field coordinator or other SMTC 
member usually is pi^^sent at the first session of each workshop. Other- 
wise, portal leaders are encouraged to contact the field coordinator when . 
they have questions or need assistance. The usefulness df this procedure 
is limited, however, by the initiative of the portal leader and his ability 
to identify and^ diagnose his needs. 

An effect^e communication) link must be establ^hed between the 
university and the portal school to assure a smoothly running operation. 
Several portal leaders and at least one science supervisor indicated 
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that*lt is very important to have one* person at the university who^is 
identified as the contact person. The* field coordinator serves this func- 
tion, but it i*s oft'en difficult to contact him' since much of his time is 
spent away from tKe university campus. Portal leaders an(J school^district 
sc'^jiknoe supervisors have suggested that'a staff, member who is regularly 
on the univ^sity campus Should also function as a contact person, ^ 

The 'only formal observatioti of the portal leader's workshop performance 
involves a questionnaire which participants complete at the end of^th^ work- 
shop. Although we have"not observed any problems in postal leaders perform- 
ing their functions ef fectivel^y , we suggest that the university develop a 
formal observation and support system for portal leaders when they are con- 
ducting their first .teacher workshop. This may include, for example, a 
• team teaching arrangement with an experienced portal leader,,^ the science 
supervisor, or a teaching assistant from the university." Although such 

' arrangements involve additional cost, they may be worth it. In school 
districts with science, supervisors, the responsibility for coordinating, 
supervising, and supporting th^ portal leaders shifts for the most part to 
the local level. In other school districts, the university must maintain 

^ a* major responsibility. ' 
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SCHOOL DISTRICT RESPONSIBILITIES IN THE PORTAL SCHOOL PROGRAM 



We have' identified three major categories of school district respon-. 
sibility which are important factors in tlya portal school program: 

\. Providing leadership and financial support. 

2. Planning and conducting workshops. * ^ - * 

ft 

3. Assessing portal school progress. 

While there is considerable interaction between the school district and f 
the university in performing these functions, we thinl^^he major respon- 
sibility for their effectiveness lies in the school district. ^ 
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PROVIDING LEADERSHIP AND FINANCIAL SUPPORT 



• The portal school is designed to provide a "portal" betv^n the 
resources of the university and the needs of the school district. 'Some 
school districts use the portal school to explore alternatives to the 
science curriculum they are using. The goal is to provide the teachers . 
wii:h aivoopportunity to survey the science, curricula that are available 
and to decide which yJfovil'd best fit their ne^ds, <= Other school districts 
use the portal school as a means of implementing 'science curricula vhich 
have been a^gpted. The goal 4s to provide ^teachers with the competency 

to use the new curricula in the classroom. » 

) 

In ther initial meetings with the field coordinator, the' schpol 
district is encouraged to ,^fine its science -instruction needs' a tid 
foriQulate objective&-^K«: participating in -the portal school program.^ 
The coordinator stresses that the^«chool district must take responsi- 
bility for^the direction of the, porta3^chool because the program* s 
strength lies in its abilit.y to be tailored to local needs. The univer- 
sity Cannot direct all of the portal schools;- school district commitment 
and leadership is essential. 

Two of the portar schools -we visited did not appear to have strong- 
school district support. In one, an elementary program, the initial 
contact fflth thfe Science and Mathematics Teaching Center (SMTC) was mad^ 
by the teachers ^themselves* The school district's central adminisTtation 
did not become involved in the program until later when a science super- 
visor was appointed. By this time several portal school workshops had • 
already been conducted. The portal school voi^kshops appeared to be 

'sim^ar to extension courses for teachers (Vith the exception that th^ 
were taught by a portal leader) rather than a* school district program for 
meeting its science education needs. There have been •nearly three years 
of exploration workshops and only recently have plans'- stanted to develop 
for making the transition from exploration^^p^mplementatJbn. The other' 

-portal school without strong school district support serves secondary , 
science teachers in^a sparsely populated region. It 'encompasses several 
school districts ^nd thus does not have onfe central administration .to 
coordinate it. Although the science supervisor in-.^f\e district in which 

•the portal school Is centered has provided strong leadership support,, 
commitments from the school- districts are not apparent.^ > ' 

Both af these portal schools are ^at the explorati£>n level, but 
neither has had' a firm plan for moving beybnd exploration. In "the .second 
ary^ regional port^ sjchool,-. it is difficult to respond to implementation 
needs at a region^ level because plans for impiement^tion are made inde- 
pendently in each school district. In^^the elementary por^l school we 
have observed the need for sttong leadership support from a central level 
.of authority, which did not exist. until recently.- It is our judgment-- 
that careful planning and a commitment of support are important prereq- 
uisites for a portal school, jj the exploration workshopjs are Intended 
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*to provide efficient facilitation of adoption decisions and the implemen- 
tation workshops are intended .to achieve worthwhile changes in science 
education, aomeone in authority must help develop the plan, organize the 
activities, and provide leadership in directing the portal' school* The 
school district's central administration must provide this leadership or 
suppor^t someone else who is providing it. In, regional portal schools, 
other ways must be found to provide strong leadership. 

In addition to providing leadership for^^e. overall plan of the . 
portal school, the schpol district must ef f ici^tvtly organize and manage 
the activities of the portal school. This includes publicity, scheduling 
of workshops, responding to ijeeds of portal leaders and teachets, pro- 
viding curriculum materials, etc. Workshops must be arratiged to miniB 
conflicts with teacher- schedules, ' Other important antecedents include 
the scheduling of portal leaders and faciliti'es and providing an adequate 
supply of curr.iculum materials for each workshop. If there are problems 
in scheduling portal leJaders, additional leaders should b^ selected and 
trained. Several portaJL leaders who have taught workshops continually 
for /two years expressed, a desire to teach fewer workshops or at least 
take '*leave" for a term. 

The scheduling of a workshop is followed by publicity to the teachers. 
Existing communication systems within school districts seem to* provide the 
main channels for pprtal school publicity, WhilB the means of communica- 
tion vary from one school district to another, most teachers reported 
hearing about the portal school from the science supervisor, the science 
department head, the principal, and through school district memos. 

Financial support by the school district is also essential for the 
operation of a portal school. The primary need is to purchase an adequate 
supply of science curriculum materials for both the workshops and the 
sfcience classes. This and other aspects of financing the portal school 
are discussed in the final, section of the report. The point we want to 
make here is that the school district must initially estimate the finan- 
cial costs of the portal school and make a^ corresponding financial com- 
mitment before the portal school is established. 
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PLANNING AND CONDUCTING WORKSHOPS 



Materials for th* 



The teacher workshops are ijhe primary means of the portal sch^l for 
•achieving changes in the science curriculum. The following topiVs, which' 
relate to planning and conducting a workshop, provide an outline for this' 
section of the report: ' . . 

• • 

1. Content in the workshops. ^ 

2. Instructional^proces^^4& In the workshops. ^ ^ 
3r . Scheduling the workshops* ^ ^ 

^w^kshops. < ' s 

5. Teacher i^otivation to participate. 

6. Teacher application^ in science classes. 

Content in the Workshops 

ft ' 

The three general levels of exploration, implementation, and creative 
expansion are useful in defining the content of a workshop. Exploration 
^ workshops at the elementary level may explore science curricula such as 
SCIS, ESS, andN^APA; secondary workshops might explore ISCS or BSCS. 
LQx:ally developed programs may al^o be included in the workshop. The - 
participants exam:^jne the^merits and shortcomings of the curricula, con- 
sidering the needs of the students and their ability to adopt the style 
of teaching promoted by the program. 

) . ^ • 

At the elementary level we observed exploration workshops which 
covered two or more curricula as well as explorations which covered only 
one science curriculum. It is claimed that a workshop on a single curric- 
ulum is an exploration, because it assists: the -school district 'in deciding 
whether to adopt the curriculum 'or continue with the current program. 
However, there are multiple science curricula which are available at the 
elanentary level, and it seems more worthwhile to include at least two 
'or three of these programs "in an exploration workshop. 

At the secondary level we founS that most exploration workshops 
include only one curriculum. Since there are a number of- content areas 
in secondary science, the alt'ernatives for each area are limited. Hence 
teachers are exploring the merits and shortcomings of a new curriculum 
in comparison to their current science program. > Although a^ stronger 'case 
can be mad6 at the secondary science level for djncluding only one curric- - 
ulum in an exploration workshop, we encourage^e portal schools to explore 
all alternatives where multiple curricula' a??6 available . 
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The science supervisor, portal leaders teachers, and the field coordi- 
nator "liaye varying degrees of influence on the content of exploration work- 
shops. For exan^le, the science supervisor or a science curriculum com- ' 
mittee may decide which curricula will be explored, and then' the portal 
leader * determines the cont ek of the individual sessions. In other cases, 
the portal leader may be responsible for. both types of decision. Teachers 
also may influence the content of the workshops; for example, the science 
supervisor in one district; periodically surveys secondary science teachers 
to determine what workshops th^y would like to have.' In addition, feedb-ack 
from participants in a workshop may suggest the content for a subsequent 
, workshop. Finally, the &klC may make suggestions for an expljj;x»|i>ion wopk-' 
shop as it did during the past year by encouraging a number of portal school 
to conduct an interdisciplinary environmental studies workshop for science 
and social studies teachers-. 

. - h 

Implementation workshops fotus 011"^ a specific curriculum. The purpose 
of the workshop is to prepare the teachers to use the adopted program 

* successfully in their science classes., topics in the implementation work- 
shops include the teaching philosophy of the program, scope and sequence 
of the program, use of materials and equipment, teaching strategies, 
student evaluation, etc. 

The portal schojjl workshops do not focus directly on teaching science 
knowledge to the teachers. However, we observed that teachers do learn 

* science knowledge by working with the curriculujp materials in the workshop. 
Several secondary teachers expressed a preference for. more science knowl- 
edge. . This issue should be handled in each portal school by first assessing 
the needs of the te'achers and then^developing a workshop plan which corre- 
sponds to the teachers' need^. Inc^most cases the portal leader or a team 

of portal leaders can effectively present the science knowledge which is \ 
needed. Occasionally it may not^e possible to' find a portal leader who is 
\epmpetent to teach a tinit or more of science knowledge that the teachers 
• ^need. The 'portal school should then explore a co-leadership arrangement 
with the SMTC whereby the portal leader would share teaching responsibil- 
ities with a university professor. Such an arrangement could provide an 
opportunity for increasi^ng advanced science knowledge without losing the 
valuable practical orientation of the workshop. ^ 

Perhaps the best way of preventing the misuse of the' exploration 
workshops is for the field coordinator to insist on a clearly specified 
plan for the exploration phase. Then a procedure could be established 
for checking each workshop proposal for its congruence with the overall 
exploration plan. The explorationxworkshops- should be a means for moving 
into the implementation phase. Ii> most, cases it should not be necessary 
for the exploration phase to continue more than one year, especially'. If 
a district-wide adoption' of a ^ngle curriculum* is planned. -The number 
of teachers whor^aTTdbfipate extensively in the exploration' activities 
depends on locsfl circumstances, ' but generally a small number of teachers 
(perha^ps 20-50) is sufficient. These teachers should then participate 
in making the adoption decision. Other .teachers could participate in 
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shorter exploratitjn workshops (equivalent to a one-credit course) to obtain 
an overview o||the curricula which are being explored by the school district. 
Where adoption decisions are decentralized, it may be necessary to schedule 
the workshops over a longer peri6d and involve more .teachers in .the explora- 
tion. -This obviously applies to the s.econdary regional portal schools where 
adoption decisions are decentralized in the school districts. 



It appears that most secondary science teachers who' have participated 
in an exploration workshop are able to implement a new curriculum without 

^ participation in an implementation workshop.. There seem to be two reasons 
for this: (1) The secondary exploration workshop in most cases is an 
intensive study of a single curriculum and hence is similar to an implemen- 
tation workshop. (2) Most secondary science teachers have extensive 
training and experience in science education whereas most elementary 
teachers have relatively little training and experience in science educa^ " 

- tion. Hence elementary teachers may accrue the greatest benefit when 
• ■ ' highest priority is placed on implementation workshops with lower priority 
on explorati/3n. We are not suggesting that Exploration should be deleted 
frojn elementary portal schools but that implementation should receive 
greater emphasis than exploration. Secondary teachers, however, generally 
prefer to explore before making a commitment,- and then they proceed with 
the implementation without formal assistance. 

The exploration ot curriculum alternatives is an^important -stage in 
the total process of making curriculum changes. " However, we are concerned 
that an individual portal school may be misused in the absence of clearly 
specified school district goals for the exploration. In two of the portal 
schools we visited, the teachers responded more frequently that they have 
participated in the workshops primarily because it is easier to get academic 
credit. Several teachers in one district reported that they have accumu- 
lated nine semester credits by repeating exploration workshops. We were 
given the impression that these portal schools were serving the personal 
needs and interests of the teachers at the Expense of attaining planned 
changes in the science curricula of the schoOi districts. We contend that 
there is considerably more payoff for both\the school district and the 
teachers at the implementation ^stage, and hence exploration should be 
restricted to a relatively shorter time interval. . ' . 

- - • i . ' 

Instructional Processes in the Workshops 

* * 

The highest priority in all workshops we x»bserved is given t'o teacher 
experiences with the curriculum materials. ' The teachers usually worfe with 
^ these materials in small groups of two to five members. When asked how 

time was spent in a workshop, participating teachers from ali^our portal 
schools mentioned the small-group activity more frequently than any other 
activity. In two portal schools some teachers' also reported that they 
worked individually with the materials. Small group discussion was also 
mentioned as a frequent activity in one portal school. Observation of 
several workshops indicated that such discussi'on was probably related to 
the experiences teachers vere having with the curriculum materials. ^ ■ " 
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The role of the portal leader also varies. The "teachers in ord ■' 
portal school indicated that the leader frequently conducted demonstra- 
tions. In. the other portal schools, however, such activity was rarely 
mentioned. Discussion involving the entire group was reporte?^as a 
frequent activity in three portal schools but not in the fourth. The ' 
leader in one portal school 6ften discusses with the group such topics 
as safety precautions, storage procedures, and grading alternatives, but 
the leader in another t)ortal school has the teachers work with curriculum 
materials during the entire session.^ Another .leader uses f ilms#and gu^t 
lecturers frequently. Sometimes a leader discusses briefly with the grdup 
some of the principles involved in the activity; in other cases the . 
teachers are left to learn the principles on their own> 

Since the new science curricula emphasize the >n^uiry me^od of 
learning and most teachers have had little or no experience wiffa the use 
of this methbd, the inquiry method has been used extensively as d^e 
teaching process in many workshops. Instead of' teaching "the participants 
about the new science curricula by lectures, demonstrations, and readi-ngs, 
the portal leaders emphasize teacher experiences with the curriculum ' 
materials. The teadhers, in effect, perform the same role that their \ 
students would perform in. science class and thus discover the concepts 
and principles of science by interaction with the materials. This process 
seems essential if the teachers are going to develop a comfortable feeling 
for the inquiry method of learning. 

' We question, however, the nearly exclusive use of the inquiry method 
by some portal leaders. First of all, use of the inquiry, method as the 
only .instructional process is' an inefficient use of workshop time. It is 
important for the teachers to have , extensive practice with the pethod of 
directed inquiry, but it restricts the amount that can be learned in- the 
short. time of a workshop. It is our judgment that the payoff was greater 
in those workshop sessions where a period (about 30-45 minutes) of inquiry 
with the curriculum materials was followed by the portal leader's (1) sue- 
^f^i^f/^ the- inquiry could be continued', (2) brief demonstrations 
■of additional activities, (3) discussion of the science content which is 
learned in the activity, (4) discussion of practical problems in using the 
materials with science classes, and (5) discussion of the relationship of 
the activity to other units in the curriculum. 

The second problem with exclusive use of the inquiry method in the 
workshops is that extensive practice of the student's role does not 
necessarily lead to an understanding of the teacher's role in using the 
inquiry process. Several teacherS whom we interviewed reported that their 
studfents were bored with the new curriculum because they finished the 
lessons in a very short time. Further questionning revealed that these 
.teachers not encourage the students to make inquiries! as is intended 
for the pif^dgram. Teachers should be given an understanding of the philos- 
ophy of instruction which is recommended for the curriculum. In addition, 
portal leaders should describe clearly the role 'of the teacher and illus- 
trate explicitly the various strategies which can be used in directing ■ 
an inquiry. Consideration should also be given to providing opportunities 
for the teachers to practice the teacher role in the workshop. 



34 . 

31 



r 



The SMTC discourages extensive assignments for the workshops and 
instead encourages the portal leaders to. stimulate the teachers to try* 
some of the curriculum material^ in tfieir classes while they are enrolled 
in the workshop*. This suggestion is generally followed in the four portal 
schools as most teachers reported that they were required to ^do little 'or 
no work between workshop sessions. The assignments they mentioned, however, 
were generally quite practical, such as observing a scien'^ce class where a 
new. curriculum is being used, trying new curriculum materials with a class, 
reading abouf a new curriculum,' reading about methods of teaching science, 
vpriting an evaluation o^ workshop activH^ties, and studying concepts and 
principles of science* On the other hand, most of the teachers reported 
voluntary activity between sessions* Of t;hfe. teachers surveyed by question- 
na,ire^ 78 percent claimed they had voluntarily tried out the new science 
materials in their class.* Other f requently'mentioned voluntary activities 
were: reading about new science curricula (58 percent), reading about 
ways of teaching science (45 percent),, and studying concepts and principles 
of science (39 petcent)* * 

Based on questionnaire data, the teachers seem satisfied with the 
processes of the workshops* ^Eighty-five percent, of the participating 
teachers who were surveyed in the fpur portal schools stated that the - 
presentations in the workshop sessions were* effective* In addition, 82 
percent of the teachers stated, that they received more than adequate 
instruction c^.how to use the materials in their classroom* Seventy per- ^ 
cent of the .tekchers were satisfied with the way the workshop sessions 
were otganized*^ 



Scheduling the Worksh 




y Most portal school wbirkshops are scheduled during the late aft^ernoon 
after classes have been dismissed* Teachers are usually exhausted at this^ 
time of day, but those interviewed felt the convenience of going from school 
directly to the workshop overrides the fatigue factor* Evenings, weekends, 
and suntoers are less desirable fb.r most teachers* Exceptions to this 
pattern, however, do exist* Since some secondary workshop participants 
in the -sparsely populated region must drive as far as 75 miles one way, 
the sessions are held in the evening. Also one secondary workshop in the 
large urban district was held during the summer. None of the teachers who 
participated in ^workshops r^ported^any dissatisfaction with the schedule* 
Only 17 percent.;of those teachers who have not participated in a workshop 
claimed that inconvenie;it* schedulin'g was a major factor for their lack of 
participation* 

- Many df the workshops are scheduled for 17 weekly sessions of two ""and 
one-half hours each. This corresponds to a three-credit semester course* 
The urban elementary portal school has also scheduled ten-week and five- 
week workshops for two artd one^redit^ respectively* These workshops 
appeal to some teachers who otherwise would not. enroll-. The shorter work- 
'shops have a^so been used to provide teachers with a second workshop ^ 
experience, this tin^e after they have practiced the ideas which were 
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learned in* the first workshop. The a^^ailability of alternative workshops 
seems to satisfy the preferences of tiachers. Of those teachers in the 
urban elementary portal school who were surveyed by questionnaire, 88 per- 
cent rated the number of weekly sessions as about right. 



One advantage of the^ortal school program is that the workshops', are 
conducted in the local school district rather than at ^a university. Some 
school districts have established a center for the portal school where 
curriculum materials are stored'and workshojps are conducted. In addition 
to the central location, the urban elementary portal school has scheduled 
the shorter workshops in schools where a minimum enrollment is assured. 
^Thus many teachers can attend a workshop in their own building, a consid- 
erable convenience for most teachers in a large urban area>. The teachers 
reported satisfaction with the location of the workshop. with 92 percent 
of the urban ^ementary teachers rating the location as convenient to 
.very convenient. m • ^ 

§ 

Even though they haW attended an implementation workshop, teachers 
still encounter unanticipated problems when they are implementing a new 
\jurriculum. Teachers can discuss these problems with other teachers in 
the school, but there seems to be a need for planned and' organized oppor- 
tunities to interact with other teachers who are using the program, 
especially teachers at the same grade level. The teachers suggested 
follow-up sessions to seek assistance with problem areas and share ideas 
for using the curriculum effectively. Teachers would not enroll in these 
sessions for credit and could pa^icipate as frequently as they *pref erred. 
We strongly endourage portal schools at the implementation stage to 
develop and adopt a plan of this t3T)e. 



Materials for the Workshops 

The science curriculum materials are a very important part of a portal 
school workshop. • The teachers use the materials in both the workshop 
sessions and their science classe^ Most teachers will be stimulated to 
try the curriculum only if an adequdta^upply of ^;mat;erials is conveniently 
available. One way of providing access ta the materials is to store them 
in a central location within the school district. Teachers can then check 
out the^ material? 'for a period of time. And. indeed teachers do checjc out 
materials regularly while they ai^ enrolled in the workshop^ if an adequate 
supply is available and the portal leaders encourage them, ^^ofilems occur, 
however, when the workshop is not conducted in the same location where the 
materials are stored or an adequate supply of materials is not av^itlable. 
These problems can be iMnimized though if the school district has made a 
commitment to, provide strong financial support "^r the portal Jschool. 

One of the school district's in the exploration' phase hae/seiected 
"pilot" schools for an extensive tryout of the .alternative science 
curricula. Although there are strong merits in this approacJji, we empha- 
size that its effectiveness is contingent upon having an ad^uate number 
of curriculum kitsyln each "pl$lot*\ school. ^ . ^ ^ 
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Another problem occurs during the implementation stage when a teacher 
has participated in a workshop, but the curriculum materials have not been 
ordered for that teacher's class, In the urban school district we portrayed, 
in^ti^U/ltional budgets are decentralized so that the new science curriculum 
is ordered at different times by various schools • Hence some teachers . ' 

the new currlWtSRpftt^ut 



Teacher Motivation to Participate 

The por^[^L school has 'failed, of course, if i:eacher's do not pa^rtici- 
^ pate in the worHtehops, Three factors seem to account for most of the 
.reasons that teacljers gave for participation, Fir«t, there is an impor- 
tant payoff in learning how to use a new curriculum and improving their 
science instruction. Teachers stressed ^the practical orientation of the 
workshops; they .learned^ things which we^e immediately applied' in their 
science classes. Related to this is the -fact that-.portal leaders are 
, fellow teachers who have experienced the thrills and) frustrations* of 
usiag the new curricula. Questionnaire data also support the educational 
v^lue of the workshops for the ^eachers as a major -Motivating factor. 
The most frequently' mentioned reasons' given for participation in the 
workshops were* to learn more about ways of teaching science and to learn 
about new science curricula. 

A second important incentive is that the teachers receive graduate 
credit ^for the workshops. Academic credit was mentioned frequently in 
both the questionnaires and the interviews as a reason for. participation 
in a workshop. This credit can then be used to meet recertif ication 
requirements, Jto raise a teacher's standing the pay scale, oi; to 
meet requiremetits for a graduate degree. Acad emif credit is most • 
important as an incentive in those portal schools where'teachers pay 
• their own tuition. 

"The scheduling of the workshops is a third factor that can motivate, 
teachers to participate.. Since the workshops are held within the school ' 
district (with the exception of regional portal schools), travel distances 
are significantly reduced ^rom what they would be for a university course. 
Even in the sparsely populated region where some teachdrs drove as far 
as^5 miles one way to attend a workshop, this distance was 'still shortep 
thah to the^ nearest university. Scheduling workshops in the afternoon 
appeals to most teachers, and the shorter workshops have provided an 
incentive for some teachers who otherwise would not participate. 

Teacher Applications in' Science Clares 

The portal school is successful to the extent that it facilitates 
. improvements in the schoot^district's science curriculum. It is ^.ntend^d 
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then that teachers will apply the new materials' and reaching .ideas in 
• their classes both during and aftei^ the workshop. ' • 

While the workshops are in session, teachers are encouraged to try 
^some of the activities with their classes. If the necessary materials 
are not available at; the teacher *s school, it Is usually possible to check 
out the materials fiom a central storage facility in t!he district or borrow 
them from the portal leader. The responsibility for actually using-the 
, materials, however, rests ^wlth the individual teacher. The portal leader 
suggests activities which teachers can try, but each t eacher mu st take the 
initiative in applying the , materials and ideas. 



From the questionnaire data, it appears that application in science 
classes is more^likely; to ^occur during elementary workshops than during 
secondary, workshops. It seems that elementary ^ teachers generally are 
eagec^o substitute an activity or unit for lessons in the "old" curric- 
ulum. \Secondary teachers' said there are problems in trying activities * 
from the new curriculum, including the organization and sequence of 
topics in the course. Some secondary teachers reported that next year ^ ^ 
they will be teaching a dif ferenl^s<^urset in which they will use the new 
curriculum, but the. materials are not appropriate for their current - 
science classes. 

Where classroom application did next occur, it /was apparently ^f or, 
reasons other than l^ck of materials. Se|enty-twdr percent of the teachers 
reported that cuj^ticulum materials were conveniently available for use in 
their classes while they were enrolled ii^the workshop. In only one , < 

^ portal school did a significant number of teachers claim the materials 
were not» available. In addition, for secondary teachers it is generally 
inconvenient .to transpo^rt from a central storage facility all the materials 
which are needed for dn activity. 

* * ' * 
After participating in an implementation level workshop, "the teacher 
.is expected to use the new curriculum in the science class. Such use, 
however, is contingent on the availability of the materials. This is " 
•where the centril administration of the school district tperhaps also the 
school princip^) is responsible' for purchasing an adequate supply of 
♦ materials so teachers are not delayed in making the Implementation. 
Delays are costly in terms of teacher motivation, teacher time required 
for reviewing when materials do arrive, and depriving students of Imme- 
diate educational benefits. ' c 



\ 



Besides the availability of materials, encouragement and support for the 
teachers are also important. Support can be provided by (1) portal leaders, 
especially for the teachers in phe same building where they teach, (2) the 
^science supervisor, and (3) science department heads or resotirce teachers. 
The urban school- district attempts to develop strong leadership in each 
schcrol for the curriculum being implemented. ^This person (s) can assist - ' 
the teachers with their implementation problems and suggest activities 
and procedures which are effective in other classes. The follow-up ^ 
sessions which were suggested in a previous section. Scheduling the ^ 
Workshops, may be a useful mechanism for the school leader to use. 
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In the portal schools that we portrayed, there is evidence that 
classroom application has occurred. The urban school district hast • 
implemented SCIS in 74 of 93 , elementary schools. Six of 18 junior high 
schools have implemented ISCS. In the sparsely populated region, a 
number of schools are using ISCS, BSCS, or Harvard Project Physics.- The 
science educator on .the visitation team reports ^hat this is an unusually 
high degree of implementation for such a remote area. Finally, 73 percent 
of the teachers in the four portal schools indicated that they are now 
using diffez5ftnt curriculum materials in their science classes as a result 
of the portol school. 

In th§* urban junior high schools where implementation of ISCS has 
not occurred, difficulties in finding storage facilities must first be . • 

overcome. Junior high schools which have implemented ISCS often differ 
from those which have not by having more space to store the » necessary 
^equipment. Low student reading level *^as also stated as a reason for 
lack of implementation in some schools. 

In addition to the use of different science, materials, changes in 
teaching methods have occurred ^as a* result "of the\portal school program. 
Eighty percent of the teachers who have participated in workshops reported 
that their students are now more actively involved ,in doing experiments 
and in working with materials. Fifty percent repeated that they now do 
^. less lecturing. Sixty-three percent o^ the p*artic±pating elementary n 
teachers are now mqre enthusiastic about^ teaching' science. Most teachers ^ 
who have not implefflen|ted a new curriculum did report in the interviews-r^ 
however, that they helve changed the style of teachingf science. 

» • 

General Conclusions About the Workshopsy * ^ . . * 

. Teachers generally are enthusiastic |about the portal school workshops. 
They like the practical orientation of the sesslbrfs and have suggested 
workshops for other curriculum areas as well.. .NItorly all teachers^who 
have participated in workshop^ rated them on tiu^juestionnaire as success*^ 
' ful to highly successful on the, following f our ■^^texia: 

1. As a means of improving science teaching ^97 percent). 

2. As a means of improving .knowledge of science (90 petcei^t) . 

3. As a means of leaming^bout new science curricula 

(98 percent) . ^ at ' , 

4. As a means ^of meeting the school d:{^^ric7*s needs in 
!^ ' science education (93L percent). ^ ' 

When asked about the outstanding features of the portal school progMto, 
71 percent of the teachers stated that they can -readily apply what is^ * 
learned. in the workshops to their science classes.^ Other ^outstanding fea- 
tures w^ich the teachers mentioned include developing a knowledge and 
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understanding of science which laany teachers have* lacked (44 percent) 
and the emphasis on good methods of teaching science in the workshops 
.(43 percent). ' 

In addition to generating teacher enthusiasm, the workshops also 
seem to accomplish their goals. Where designed to implement a science 
curriculu^^ this implementation is, in fact, occurring. 
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ASSESSING PORTAL SCHOOL PROGRESS 

After the school district has identified its science education needs, ' 
it plans and puts into operation a portal school for responding to the 
needs. At this point the school district should begin to assess the 
progress which is being made in attaining the goals which were established 
for the portal school. In this section we are suggesting ways in which 
the school district might develop a pi^cedure for assessing progress. 

Consider the following examples of, portal school goals: 

1. By May 1974, (a) ^out 25 elementary teachers will complete 
an extensive "pilot'* study of the ESS, SAPA, «id SCIS pro- . 
^ grams, and (b) all elementary teachers will have the oppor- 
tunity -to participate in exploration workshops on the ESS, 
SAPA, and SCIS programs and*to try some of the activities 
in their science classes, 

^ 2. ,By February 1974, Levels I and II of the ISCS program will 
be implemented in all eighth grade ' science classes. 

3. By September 1975, about ten high school science and 
social studies teachers will complete the development 
^of an environmental studies curriculum. I 

Siru;^ each of these goals is stated as a terminal outcome, the next step 
is to formu]fete intermediate checkpoints. Suppose, for the second goal, 
that there are ten junior high schools in the, school district. The portal 
school was started in June 19^2 with the intent that all eigtith grade ^ 
science teachers within a school would make the implementation at the 
same time. A form like tl>e example in Table 4 ma^^be used to record the 
progressSin attaining the' goal. An X ^indicates that implei^ntation has 
occurred in the school by the date shown at the top of the\olumn. The 
report sho^s that the school district is maintaining its schedule for 
implementing the ISCS curriculum. 

^ Iti Table 5 we have displayed a sample form for the first goal but 
without listing the names of all schools in the district. The numbers - 
in the September 1973 column indicate the number of teachers who partici- 
pated in the first workshop and are now doing the "p^lot" study. The 
s^ibsequent columns represent an accumulation of teachers from each school 
who have fompleted the er^^ploration workshops. In this illustration we 
have not specified performance criteria because the goa^l is to 'provide 
teachers with the opportunity to explore. It is not intended that all 
teachers will explore the three curricula. The use of performancf criteria ^ 
as in Table 4, are more appropriate during the implementation stage. The 
form is used then to record progress in an objective manner. This data 
can be ui&ed by the director of, the portal school to * identify schools where 
few^or no} teachers have participated. 
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TABLE 4 

SAMPLE FORM FOR REPORTING PORTAL SCHOOL PROGRESS 
OE ISCS IMPtEMENTATION 



School 



Blake 

Centennial 

Central 

Dexter 

Lakevlew 

Mann 

Oakdale 

Portal 

'Trimble 

Valley 



Sept. 1972 



Feb. 1973 



Sept. 1973 



GOAL: 
PROGRESS: 



X 

4 
5 



8-' * 



Feb. 1974 



10 



ft 



r 



o 
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^ , TABLE 5 

- SAMPLE FORM FOR REPORTING PORTAL SCHOOL PROGRESS 
• , OF ELEMENTARY SCI^CE EXPLORATION , 



School 


Number of 
"Science , 


Number of 


.Participating Science Teachers 


Teachers 


Sept. 1973 


Dec. 1973 Mar. 1974 May 1:974 




D 
0 






Columbine 


14 




10 


^ Fores ^ 


. 7 ' 


0 


. 0 


Freedom 


. 22 


■ 7 


■ 7 


Greerrfield 
• 
• 


20 

• 


7 

• 


15. ' 

• ' • • • 


Whittier 


11 


_0 


_! ' 


I 


- PROGRESS: 


- 2^ 
• O 


V 

84 
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^ In addition to the record forms described previously, the following 
types of data could be collected: 

1. Teacher d^ta on the workshops. The SMT^ has ^een admin- 
istering a questionnaire of this type to* participants at 
the end of each workshop to evaluate the Ijuality of the 
workshop-. 



2. Teacher data on applications ip science classes. A ques- 
tionnaire could be given to teachers periodically to assess 
the changes they have made in the science curriculum and 
the problems they are having. 

3. Student data jpn applications in science classes. A ques- 
tionnaire cduld be given to students to obtain their 
observations of cjianges in the science curriculum and the 
impact of the portal school program. ^ 

t 

We^ do not recommend a complex system for assessing portal school 
progress. But seV^ral simple procedures for systematic data collection 
may enable the school district to operate a more efficient portal school 
with perhaps a higher quality of outcome. 
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' ' FINANCING THE PORTAL SCHOOL PROGRAM 

I 

The financing ,of the portal school program involves the five types 
of expenditure listed on the left side of the matrix in Figure 1 and the 
four sources of money listed on the bottom side. A "yes" in the matrix 
indicates the actual source of funds (bottom of column) for each major 
category of expense (left side) in the University of Wyoming *s portal 
school program. This section of the report contains a description of the 
expenditures and financial resources with judgments about financing the 
^ portal school program. 



EXPENDITURES 



A. 


Field 

Coordination 








yes 


B. 


Development of 

University 

Services 


yes 




yes 


1 


6 

c. 


Training 
Portal Leaders 






yes 


yes 


D.' 

/ 

E. 


Salary for 
Portal Leaders , 






yes 




Curriculum 
Materials 


« 


yes 










University 


School 
District 


Tuition 


NSF 



FINANCIAL RESOURCES 

^ , / 

Figure 1. Matrix of Expenditures and Financial Resources in the 
Univ*sity of Wyoming's Portal School Program 
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Expenditures 



Money is one of the important resources for implementing and main- 
taining the operation of a portal school program. Money is needed to: 
(A) provide personnel for the coordination of the program between the 
university and the school districts, (B) facilitate the development of 
university services for the program, (C) train the portal leaders, 
(D) provide a salary for the workshop leaders, and (E) purchase materials 
for workshop activities and teacher tryout in the science classes. In 
the following section we describe briefly the warys in which these expen- 
ditures are used. ^ 

A. The coordination of the portal school program between the ' 
university and the school districts is carried out primar;Lly by the 
field coordinator. The outstanding progress which has been made in 
starting portal- schools was facilitated extensively by his previous ' 
state department experience in both Colorado and ^Wyqmings A person 
with less experience in the region probably woifld have made slower 
progress r The scope of the coordination task depends oil factors such 
as Mze of the region, number of school districts in the region, whether 
the^rogram focuses on both elementary and secondary curricula or only 
one o^ the levels, and the number of school districts wilh science 
supervisors. 

B- The services proy^ed by the .Science and Mathematics Teaching 
Center (SMTC) are an important component in the portal school program. « 
The resources for these services include- the stkff , facilities, and^ 
materials of the SMTC. These resources are most obviously used in the 
training of portal leaders. In addition, the staff mu&t be released to 
some extent from their other responsibilities so they c'an be available 
to assist school districts directly with their portal school activities. 
There is also a ne^d for the SMTC to use staff time for developing 
instructional units, demons trati^ig the curriculum materials to teachers 
who visit the center, and conducting research and evaluation activities. 
Although the form^ in which the university resources are organized^ and 
the services are delivered will vary with institutions, poney will still 
be needed to support the. development of the services. 

C. The training and certification of portal leaders ar« critical 
functions in the portal school program. . In addition to covering the 
expenses of the minitute, it is probably necessary to offer a stipend to 
the teachers who attend the leadership training institute. Many of the 
teachers who are nominated to be portal leaders are not motivated by the 
opportunity to obtain additional un:}.versity credits. Furthermore, they 
take on additional responsibilities after being certified' as portal 
leaders. Since the salary of portal leaders is relatively small, some 
.incentive, e.g., a minitute stipend, would help to attract some out- 
standing science teachers to the program. # . 

^ D. At pfe^nt the salary of workshop leaders is obtained through 
the tuition that is paid -by the teachers. /The salary is 30 percent of 
the workshop tuition. However, this is. often inadequate, especially if 
there are a small^ number of participants in a workshop. When there are 
25 or more participants in a workshop, the portal, leaders generally 
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appear to be satisfied with the salary. However, the most appealing 
workshops for some taachers.are the' short workshops conducted within the 
school. Thirty perc^t of the tuition for tWse workshops does not at 
all represent adequate remuneration foi* ck^^portal leaders. WithiM^e 
current policy, the only w^y to adequately cover the salary of teacners 
is to conduct large workshops,, and these generally are not the most 
effective workshops for all teachers. In one school district, the salary 
of the portal leader is supplemented by using money in the reserve fund. 

E. Curriculum materials are* needed for. both activities during the 
workshop and teacher tryout in the science classes. While materials seem 
to be adequate for workshop activities, problems have occurred regarding . 
the availability of materials for teachers to try in their classes. The 
urban school district has furnished a storage room with elementary science 
curriculum materials where teachers^, can check out materials. Since the 
storage room is in the same building where most of the workshops are 
conducted, many of the teachers check out netf materials each yeek while 
they are enrolled in th^ workshop. Some science kits are stored in 
schools, and thereby thh use of materials is ^ even more convenient for 
teachers in that school. The extent to wtjich teachers try the science 
materials with their classes depends on the^tonvenience of the materials, 
and convenience seems to be related to the amount of materials provided 
by the school district (a significant cost factor) and the location of the 
mg^terlals. f ; 



Financial Resources y 

The current financial resources for the portal school program include 
t|ie: (1) universlhy budget, (2) school district budgets, (3) tuition from 
the portal school workshops, and (4) National Science Foundation grant. 
The functions supported by each of these financial sources are described 
in the next section. 

1. The university budget supports B, tHe development of university 
services which are provided by the SMTC. The university should continue 
to support this function since it is an integral^part 'of the SMTC and ' 
serves preservice, inservice, and graduate education programs. ' 

2. ' School district budgets support E, the purchase of material© for 
workshop activities and teacher tryout in th^ science classes. Unfortu- 
nately, there is reluctance in some school district's to purchase science 
kits at the exploration stage. At the implementation stage, schdol 
districts may provide greater financial support for science kits because " 
a particular program ha^ been adopted. Since it is important Jthat 
exploration precede Implementation, some extjerrnal funds may be needed ' 
in certain school districts> ^especially smaller ones, to purchase science 
kits at "thfe exploration stage.- - '< 

. f • . 

3. ^The tuition for the^workshop is-used^ as follows: (a) 30 percent 
is used for salaries of por^l leaders; (bX 30 percent is placed into a 
reserve fund for future por^l school expenses; and (c) 40 percent is y 
placed into the revolving fund of the SMTC. The reserve fund may be used/ 
by school districts to pay tuition of future wotlcshops (e.g., creative / 
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expansion workshops), supplement portal leader salaries, pay expenses for 
training additional portal leaders, purchase materials for^ workshop activ- 
ities, etc.^ The reserve fund provides a strong incentive for the school 
district to continue*^the portal school^ program. The revolving fund is 
used to support B, the development of university services for tKe portal 
schpol program; this includes a very small fee for maintaining formal ' ^ 
unive^rsity records through the extension division. The coordinator of 
the portal school program has reported that tMchers pay the full amount 
of tuition in about half of the portal schools.X^JIn the other half, the 
school district pays part or all of the tuition for the teachers. Some 
teachers who have not participated in fhe workshops said they do not need 
university credits and others reported that they would wait another year 
or two until they needed credits for recertif icatioij. Hence we encourage 
school districts to provide workshop opportunities at reduced cost- for 
teachers who do not need university credit so they will also have the 
incentive to develop the competency ^f or implementing* the new science 
^.curricula. . , ^ 

4.' The grant from the National Science Foundation is useM for A, 

at ion of the program between the university and the school districts, 
ining workshop leafl^rs. The field coordination expenses include 
ary of the field coordinator and travel expenses. 




coor( 
and C, 
both the s' 



General Conclusions About Financing 

' . ^, ' ' > 

The portal school program is able to operate with relatively ' little 
external funding. This is due primarily to the large amount of tuition 
money which supports the program and the relatively smaller salaries for 
portal leaders (as compared to university instructors who teach extension 
courses). In the Wyoming project, all of- the tuition goes directly to 
the support of the portal school programi This is not likely to occur 
at 6ther universities where a large proportion of the. tuition might be 
used for extension services and- university overhead. Hence, other insti- 
tLutions should not anticipate the same uses of the tuition money that . 
the University of Wyoming has enjoyed unless changes (or exceptions) can 
be made in university policy. 

Other institutions should not anticipate that the portal school pro- 
gram can be supported without funds from external sources, as some have 
suggested. Although the amount of external funds has been relatively 
small, it would be difficult for the school districts to absorb additional 
expenses since the level of expenditure 'for portal leaders (botA salary ^ 
and minitute expense) is already below a satisfactory level. Fu^rthermore, 
since other institutions are not likely to have. the advantages of retain- 
ing the full amount of tuition for direct expenses of the portal school 
program, as ^he SMTC has, the need for external funding will increase. 
State departments of education are another source of funding which should 
be explore^,. 
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PORTRAYALS OF FOUR P0RTAL_SCHO0]3\^ 



We ha^ ppepared descriptions of the four portal schools which were 
included in the evaluation study. The portrayals include a summary of 
the activities which have^been conducted as part of the portal school plan 
These portrayals .are intended for persons who find some appeal in the 
portal school program ^and are motivated to explore the concept further by 
•examining the-M^peration of specific portal schools . One or more of the 
four portrayals may be requested. 

Portrayal A - Elementary^ortal schopl in urban school 
district ^0Ch 93,000 students. , 

Portrayal* B - Elementary portal school in medium-sized 
sc^iool district with 14,000 students. 

Portrayal C - Secondary portal school in urban school 
district with 93,000 students. 

Portrayal EL - Secpndary portal school in sparsely populated 
region with eight school districts and 11,000 
'student^. 

Copieq of the portrayals can be^^ebtained from: 

• V . ^ 

Wayne W. Welch, Director * 
• ^ Minnesota Evaluation Project • 

College of Education ' 
/ r University of Minnesota 

Minneapolis, Minnes^ota 55455 
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